Virtual Machine Resource Allocation for Service Hosting on Heterogeneous Distributed Platforms

International audienceWe propose algorithms for allocating multiple resources to competing services running in virtual machines on heterogeneous distributed platforms. We develop a theoretical problem formulation and compare these algorithms via simulation experiments based in part on workload data supplied by Google. Our main finding is that vector packing approaches proposed in the homogeneous case can be extended to provide high-quality solutions in the heterogeneous case, and combined to provide a single efficient algorithm. We also consider the case when there may be bounded errors in estimates of performance-related resource needs. We provide a heuristic for compensating for such errors that performs well in simulation, as well as a proof of the worst-case competitive ratio for the single-resource, single-node case when there is no bound on the error

Research Software Engineers: State of the Nation Report 2017

Most research would be impossible without software, and this reliance is forcing a rethink of the skills needed in a traditional research group. With the emergence of software as the pre-eminent research tool used across all disciplines, comes the realisation that a significant majority of results are based, ultimately, on the skill of the experts who design and build that software. The UK has led the world in supporting a new role in academia: the Research Software Engineer (RSE). This report describes the new expert community that has flourished in UK research, details the successes that have been achieved, and the barriers that prevent further progress

On Dijkgraaf-Witten Type Invariants

We explicitly construct a series of lattice models based upon the gauge group $Z_{p}$ which have the property of subdivision invariance, when the coupling parameter is quantized and the field configurations are restricted to satisfy a type of mod-$p$ flatness condition. The simplest model of this type yields the Dijkgraaf-Witten invariant of a $3$-manifold and is based upon a single link, or $1$-simplex, field. Depending upon the manifold's dimension, other models may have more than one species of field variable, and these may be based on higher dimensional simplices.Comment: 18 page

Improving Simulations of MPI Applications Using A Hybrid Network Model with Topology and Contention Support

Proper modeling of collective communications is essential for understanding the behavior of medium-to-large scale parallel applications, and even minor deviations in implementation can adversely affect the prediction of real-world performance. We propose a hybrid network model extending LogP based approaches to account for topology and contention in high-speed TCP networks. This model is validated within SMPI, an MPI implementation provided by the SimGrid simulation toolkit. With SMPI, standard MPI applications can be compiled and run in a simulated network environment, and traces can be captured without incurring errors from tracing overheads or poor clock synchronization as in physical experiments. SMPI provides features for simulating applications that require large amounts of time or resources, including selective execution, ram folding, and off-line replay of execution traces. We validate our model by comparing traces produced by SMPI with those from other simulation platforms, as well as real world environments.Une bonne modélisation des communications collective est indispensable à la compréhension des performances des applications parallèles et des différences, même minimes, dans leur implémentation peut drastiquement modifier les performances escomptées. Nous proposons un modèle réseau hybrid étendant les approches de type LogP mais permettant de rendre compte de la topologie et de la contention pour les réseaux hautes performances utilisant TCP. Ce modèle est mis en oeuvre et validé au sein de SMPI, une implémentation de MPI fournie par l'environnement SimGrid. SMPI permet de compiler et d'exécuter sans modification des applications MPI dans un environnement simulé. Il est alors possible de capturer des traces sans l'intrusivité ni les problème de synchronisation d'horloges habituellement rencontrés dans des expériences réelles. SMPI permet également de simuler des applications gourmandes en mémoire ou en temps de calcul à l'aide de techniques telles l'exécution sélective, le repliement mémoire ou le rejeu hors-ligne de traces d'exécutions. Nous validons notre modèle en comparant les traces produites à l'aide de SMPI avec celles de traces d'exécution réelle. Nous montrons le gain obtenu en les comparant également à celles obtenues avec des modèles plus classiques utilisés dans des outils concurrents

State Sum Models and Simplicial Cohomology

We study a class of subdivision invariant lattice models based on the gauge group $Z_{p}$, with particular emphasis on the four dimensional example. This model is based upon the assignment of field variables to both the $1$- and $2$-dimensional simplices of the simplicial complex. The property of subdivision invariance is achieved when the coupling parameter is quantized and the field configurations are restricted to satisfy a type of mod-$p$ flatness condition. By explicit computation of the partition function for the manifold $RP^{3} \times S^{1}$, we establish that the theory has a quantum Hilbert space which differs from the classical one.Comment: 28 pages, Latex, ITFA-94-13, (Expanded version with two new sections

Lattice-based scheduling for multi-FPGA systems

Accelerators are becoming increasingly prevalent in distributed computation. FPGAs have been shown to be fast and power efficient for particular tasks, yet scheduling on FPGA-based multi-accelerator systems is challenging when workloads vary significantly in granularity in terms of task size and/or number of computational units required. We present a novel approach for dynamically scheduling tasks on networked multi-FPGA systems which maintains high performance, even in the presence of irregular tasks. Our topological ranking-based scheduling allows realistic irregular workloads to be processed while maintaining a significantly higher level of performance than existing schedulers.Postprin

Towards Scalable, Accurate, and Usable Simulations of Distributed Applications and Systems

The study of parallel and distributed applications and platforms, whether in the cluster, grid, peer-to-peer, volunteer, or cloud computing domain, often mandates empirical evaluation of proposed algorithm and system solutions via simulation. Unlike direct experimentation via an application deployment on a real-world testbed, simulation enables fully repeatable and configurable experiments that can often be conducted quickly for arbitrary hypothetical scenarios. In spite of these promises, current simulation practice is often not conducive to obtaining scientifically sound results. State-of-the-art simulators are often not validated and their accuracy is unknown. Furthermore, due to the lack of accepted simulation frameworks and of transparent simulation methodologies, published simulation results are rarely reproducible. We highlight recent advances made in the context of the SimGrid simulation framework in a view to addressing this predicament across the aforementioned domains. These advances, which pertain both to science and engineering, together lead to unprecedented combinations of simulation accuracy and scalability, allowing the user to trade off one for the other. They also enhance simulation usability and reusability so as to promote an Open Science approach for simulation-based research in the field.L'étude de systèmes et applications parallèles et distribués, qu'il s'agisse de clusters, de grilles, de systèmes pair-à-pair de volunteer computing, ou de cloud, demandent souvent l'évaluation empirique par simulation des algorithmes et solutions proposés. Contrairement à l'expérimentation directe par déploiement d'applications sur des plates-formes réelles, la simulation permet des expériences reproductibles pouvant être menée rapidement sur n'importe quel scénario hypothétique. Malgré ces avantages théoriques, les pratiques actuelles en matière de simulation ne permettent souvent pas d'obtenir des résultats scientifiquement éprouvés. Les simulateurs classiques sont trop souvent validés et leur réalisme n'est pas démontré. De plus, le manque d'environnements de simulation communément acceptés et de méthodologies classiques de simulation font que les résultats publiés grâce à cette approche sont rarement reproductibles par la communauté. Nous présentons dans cet article les avancées récentes dans le contexte de l'environnement SimGrid pour répondre à ces difficultés. Ces avancées, comprenant à la fois des aspects techniques et scientifiques, rendent possible une combinaison inégalée de réalisme et précision de simulation et d'extensibilité. Cela permet aux utilisateurs de choisir le grain des modèles utilisés pour ses simulations en fonction de ses besoins de réalisme et d'extensibilité. Les travaux présentés ici améliorent également l'utilisabilité et la réutilisabilité de façon à promouvoir l'approche d'Open Science pour les recherches basées sur la simulation dans notre domaine

Incorporation of a Dietary Omega 3 Fatty Acid Impairs Murine Macrophage Responses to Mycobacterium tuberculosis

by creating an immunosuppressive environment. We hypothesized that incorporation of n-3 PUFA suppresses activation of macrophage antimycobacterial responses and favors bacterial growth, in part, by modulating the IFNγ-mediated signaling pathway.. The fatty acid composition of macrophage membranes was modified significantly by DHA treatment. DHA-treated macrophages were less effective in controlling intracellular mycobacteria and showed impaired oxidative metabolism and reduced phagolysosome maturation. Incorporation of DHA resulted in defective macrophage activation, as characterized by reduced production of pro-inflammatory cytokines (TNFα, IL-6 and MCP-1), and lower expression of co-stimulatory molecules (CD40 and CD86). DHA treatment impaired STAT1 phosphorylation and colocalization of the IFNγ receptor with lipid rafts, without affecting surface expression of IFNγ receptor. in response to activation by IFNγ, by modulation of IFNγ receptor signaling and function, suggesting that n-3 PUFA-enriched diets may have a detrimental effect on host immunity to tuberculosis

Spatial Data Analytics of Mobility with Consumer Data

Consumer data arising from the interaction between customers and service providers are becoming ubiquitous. These data are appealing for research because they are frequently collected and quickly released; they cover a wide variety of attitudes, lifestyles and behavioural characteristics; and they are often dynamically replenished and longitudinal. It is demonstrated that consumer data can make important contributions to understanding problems in transport geography and in solving applied problems ranging from migration, infrastructure investment and retail service provision to commuting and individual mobility. However more effective exploitation of these data depends on the construction of bridges to allow greater freedom in the transfer of data from the commercial to the academic sector; it requires development of frameworks for privacy and ethics in the secondary use of personal data; and it is contingent on the emergence of effective strategies for the amelioration of selection bias which impairs the quality of many consumer data sources

Dynamic fractional resource scheduling for cluster platforms

Ph.D. University of Hawaii at Manoa 2010.Includes bibliographical references.This research focuses on the problem of job scheduling on homogeneous computational clusters. Clusters are widely used today for a variety of purposes, including high-performance scientific computing and Internet service hosting. While clusters may have impressive aggregate performance metrics, they are really only collections of fairly modest machines, which makes scheduling jobs for the best performance a non-trivial problem. Most clusters also need to be shared among users to amortize their start-up and maintenance costs, and ensuring that these users are treated fairly further adds to the difficulty. Existing approaches to scheduling attempt to address both of these issues, but have several limitations. We propose a novel approach, called Dynamic Fractional Resource Scheduling (DFRS), to sharing homogeneous cluster computing platforms among competing jobs. The key features of DFRS are that it leverages existing virtual machine technology in order to share resources more efficiently and it defines and optimizes a user-centric metric that captures notions of both performance and fairness. In this dissertation we explain the principles behind DFRS and its advantages over the current state of the art, develop a theoretical model of resource sharing, design heuristics to optimize the proposed metric within the given framework, implement and run simulations comparing DFRS to traditional approaches using popular and accepted performance metrics, and finally develop and test a prototype implementation based on existing technologies. Our results show that it is possible to develop heuristic algorithms that give results reasonably close to theoretical bounds for a variety of cases, that resource requirements are well within the capabilities of modern systems, and that for some scenarios DFRS can provide orders-ofmagnitude levels of improvement in performance over current approaches